Pinging @elastic/es-data-management (Team:Data Management)

We also talked about attempting to get lost operations (due to node restart). Did you look at how the existing chart for that works? Is it grepping the logs? Do you know where that logging is done? I'm wondering whether we want to try to figure out how to distinguish remote vs local in there, too.

Good question, I assume you are referring to the Reindexing tasks failures logged during shutdown graph, I think it is searching for this log, judging from the search query:

         {
          "match_phrase": {
            "log.logger": "org.elasticsearch.node.ShutdownPrepareService"
          }
        },
        {
          "match_phrase": {
            "log.level": "WARN"
          }
        },
        {
          "match_phrase": {
            "message": "*reindex*"
          }
        }

We probably need to implement something else to capture remote/local there, I can take a deeper look on that later.

We also talked about attempting to get lost operations (due to node restart). Did you look at how the existing chart for that works? Is it grepping the logs? Do you know where that logging is done? I'm wondering whether we want to try to figure out how to distinguish remote vs local in there, too.

Good question, I assume you are referring to the Reindexing tasks failures logged during shutdown graph, I think it is searching for this log, judging from the search query:

         {
          "match_phrase": {
            "log.logger": "org.elasticsearch.node.ShutdownPrepareService"
          }
        },
        {
          "match_phrase": {
            "log.level": "WARN"
          }
        },
        {
          "match_phrase": {
            "message": "*reindex*"
          }
        }

We probably need to implement something else to capture remote/local there, I can take a deeper look on that later.

Yes, that's what I was referring to. Thanks.

We also talked about attempting to get lost operations (due to node restart). Did you look at how the existing chart for that works? Is it grepping the logs? Do you know where that logging is done? I'm wondering whether we want to try to figure out how to distinguish remote vs local in there, too.

Good question, I assume you are referring to the Reindexing tasks failures logged during shutdown graph, I think it is searching for this log, judging from the search query:

         {
          "match_phrase": {
            "log.logger": "org.elasticsearch.node.ShutdownPrepareService"
          }
        },
        {
          "match_phrase": {
            "log.level": "WARN"
          }
        },
        {
          "match_phrase": {
            "message": "*reindex*"
          }
        }

We probably need to implement something else to capture remote/local there, I can take a deeper look on that later.

Yes, that's what I was referring to. Thanks.

Looking at the code, I don't think it's going to be straightforward to get the remote/local info in there. The code you linked is generic task-related code, and we are only identifying reindex tasks via a regex on the task name. I assume that changing the task name so that it included remote would be somewhat involved, and might be risky if other stuff is dependent on the naming convention (not sure whether it is).